TITLE OF THE INVENTION 
[000 1 ] Color calibration for color bar codes. 

FIELD OF THE INVENTION 
[0002] The present invention relates to color calibration and classification generally and 
to such for color bar codes in particular. 

BACKGROUND OF THE INVENTION 
[0003] Color bar codes are known in the art, though they are not prevalent. US Patents 
5,426,289 to Kinoshita et aL, 5,992,748 to Takahashi et al, and 5,869,828 to Braginsky 
teach different kinds of color bar codes. 

[0004] Typically, a color bar code system creates color bar codes using a set of set of N 
of bar code colors. For example, N might be 5. To make the code, each bar code color is 
associated with a numerical value and a color value, where the color value is in any multi- 
dimensional color space, such as the red, green and blue (RGB) space or the cyan, magenta, 
yellow and black (CMYK) space. If there are N bar code colors, the bar code that uses that 
color scheme represents numbers in base N. One exemplary code (in the RGB color space) 
might be the following: 
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[0005] In this example, there are five bar code colors and thus, the code is in base 5. 
Thus, a color bar code of the sequence [black, blue, green, yellow] represent the number 
0124 in base 5 which translates to the base 10 number 0*125+1*25+2*5+4 = 39. 
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[0006] When printing a particular bar code, the color values of the various regions in the 
bar code are sent to the printer which, in turn, prints the color bar code. After printing, the 
color bar code is placed onto an item to be labeled. A color bar code reader is typically a 
color camera that includes an image sensor, such as a CCD (charge coupled device) or a 
CMOS (complementary metal oxide semiconductor) camera, and a relevant optical system. 
The reader reads the bar code and the bar code system identifies the item fi-om the bar code. 
One exemplary color bar code system is discussed in PCT Publication WO 00/04711 to 
Shniberg et al., assigned to the common assignee of the present invention, which disclosure 
is incorporated herein by reference. 

[0007] Unfortunately for color bar codes, a given color is not sufficiently distinct for 
classification fi'om the image acquired by the reader, especially since the color perceived by 
the camera is affected by several parameters, such as the illumination and the spectral 
sensitivity of the sensor. This is discussed in the book by Wyszecki G. and Stiles W. S., 
Color Science: Concepts and Methods, Quantitative Data and Formulae, John Wiley & 
Sons, 2000, pp 117-127. histead, colors are dependent on a multitude of effects. For 
example, lighting conditions affect colors significantly; as is well known, a white stripe will 
look purplish in ultraviolet (UV) light, slightly yellow in yellow Ught and slightly blue in 
blue light. 

[0008] Thus, although bar code printer may have received the following bar code 
sequence for item 39 in the RGB color space: [(0,0,0), (0,200,255), (0,255,0), (255, 255,0)], 
the reader might produce RGB values [(10,15,15), (10,220,235), (5,200,5), (245, 245,10)] 
in the locations of the image associated with the color bar code. None of the image colors 
are the RGB colors which were printed and thus, determining which code value was read is 
difficult. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0009] The subject matter regarded as the invention is particularly pointed out and 
distinctly claimed in the concluding portion of the specification. The invention, however, 
both as to organization and method of operation, together with objects, features, and 
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advantages thereof, may best be understood by reference to the following detailed 
description when read with the accompanying drawings in which: 

[0010] Fig. 1 is a schematic illustration of a color bar code system, constructed and 

operative in accordance with the present invention; 

[0011] Fig. 2 is a schematic illustration of a color calibration system, useful in the 
system of Fig. 1; 

[0012] Fig. 3 is a schematic illustration of a color chart, useful in the system of Fig. 2; 
[0013] Fig. 4A is a schematic illustration of an exemplary output of a camera reader of 
Fig. 2; 

[0014] Fig. 4B is a schematic illustration of calculations performed on the output shown 
in Fig. 4A; 

[0015] Fig. 5 is a schematic illustration of a three-dimensional plot of exemplary data 
received from the camera reader of Fig. 2; 

[0016] Figs. 6A, 6B, 6C and 6D are graphical illustrations of slices of a three- 
dimensional color calibration LUT which is an output from the system of Fig. 2; 
[0017] Fig. 7 is a flow chart illustration of a color classification method forming part of 
the system of Fig. 2; and 

[0018] Fig. 8 is a flow chart illustration of a k-nearest neighbor calculation performed in 
the method of Fig. 7. 

[0019] It will be appreciated that for simplicity and clarity of illustration, elements 
shown in the figures have not necessarily been drawn to scale. For example, the dimensions 
of some of the elements may be exaggerated relative to other elements for clarity. Further, 
where considered appropriate, reference numerals may be repeated among the figures to 
indicate corresponding or analogous elements. 

DETAILED DESCRffTION OF THE INVENTION 
[0020] In the following detailed description, numerous specific details are set forth in 
order to provide a thorough understanding of the invention. However, it will be understood 
by those skilled in the art that the present invention may be practiced without these specific 
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details. In other instances, well-known methods, procedures, and components have not been 
described in detail so as not to obscure the present invention. 

[0021] Reference is now made to Fig. 1, which illustrates a color bar code system using 
a color association unit 30. A camera reader 32 may read one or more color bar code labels 
33, externally produced by a color production device 34 and pasted onto boxes 36 at some 
appropriate time. One exemplary color production device, used in the following description, 
might be a color Inkjet or laser printer 34. Other color production devices might be a color 
copying machine. All are incorporated into the present invention. 

[0022] Camera 32 may provide its output to a system controller 38, having an identifier 
39 and color association unit 30. Identifier 39 may utilize color association unit 30, such as 
a lookup table (LUT), to determine which colors were imaged by camera 32. In accordance 
with a preferred embodiment of the present invention, color association unit 30 may 
calibrate a color space to the range of colors that camera 32 may be expected to produce 
given at least one environmental condition m which it operates. In accordance with a 
preferred embodiment of the present invention, color association unit may associate each 
point in the color space with one of the N bar code colors of the current bar code scheme 
based on the range of colors that camera 32 may be expected to produce. 
[0023] The environmental conditions may be lighting conditions, camera conditions or 
printed color conditions. For example, camera 32 may have a wide angle lens and may 
operate only in artificial Ught and printer 34 may be an old, heavy duty printer which cannot 
produce bright colors.. 

[0024] The remaining operations to decode bar code labels 33 may be as described in 
PCT Publication WO 00/0471 1 to Shniberg et al., mentioned hereinabove. 
[0025] Reference is now made to Fig. 2, which illustrates an exemplary calibration 
system 40 for color bar code systems, such as that of Fig. 1, and its operation with a printer 
52 and a camera reader 54. In this embodiment, calibration system 40 may generate a color 
calibration LUT 94 to act as color association unit 30. Calibration system 40 may comprise 
a color chart generator 42, a color calibrator 44 and an optional color classifier 46 and may 
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operate to calibrate a color space to the range of colors that camera 54 may be expected to 
produce given at least one environmental condition in which it operates. 
[0026] Color chart generator 42 typically may generate a listing of color values for a 
plurality of color patches to be printed by printer 52 onto a substrate, such as paper. As 
shown in detail in Fig. 3, to which reference is now briefly made, the color chart, labeled 
56, may have patches 58 of color, where each patch 58 may have one of the N bar code 
colors. Patches 58 may be placed next to each other, in one embodiment, with no 
delineating lines. Generator 42 may receive from a user the number N of bar code colors 
and the RGB of each of those colors. Generator 42 may have a predefined patch 
arrangement for every color scheme size N (i.e. one arrangement for 6 bar code colors, 
another for 7 bar code colors, etc). Such a placement may be random with the condition that 
each bar code color neighbors the N-1 other bar code colors at least once. The latter 
condition may enable calibration system 40 to capture any changes of viewed color due to 
the nearness of other colors. 

[0027] Color chart 56 may be of any appropriate size for camera 32. In one embodiment, 
color chart 56 is on A4 paper and has 25 rows of 20 patches each. Patch generator 42 may 
provide the color chart information to color calibrator 44 and may provide printer data to 
printer 52 to produce color chart 56. 

[0028] Camera 54 may then view color chart 56 and may generate an image of color 
chart 56. Since colors appear different under different conditions, in accordance with a 
preferred embodiment of the present invention, camera 54 may view color chart 56 
repeatedly, each time providing its image output to color calibrator 44. 
[0029] The different conditions may be any condition that may occur during the 
expected operation of color bar code system 50 (formed of camera 54 and printer 52). Thus, 
the lighting conditions 60 may be varied. For example, camera 54 may read color chart 56 
in natural light 62 or under incandescent light 64. Other lighting conditions may include 
neon light, fluorescent light, cloudy light, direct sunlight, incandescent light, gas discharge 
lamps, such as Xenon or Mercury lamps, etc. 
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[0030] The camera conditions may be varied. Thus, camera 54 may view color chart 56 
in focus or out of focus (as indicated by arrow 66) and at various distances from color chart 
56 (as indicated by arrow 68). 

[0031] The color chart conditions may be varied. This may include rotating color chart 
56 in any direction, as indicated by arrows 70. Furthermore, the inks used by printer 52 may 
change over time. As this is not easily reproduced, patch generator 42 may produce multiple 
color charts 56, each with a different amoimt of noise in the color values. 
[0032] Typically, the conditions chosen for the calibration procedure are those most 
likely to occur during the regular operation of color bar code system 50. Thus, if system 50 
will always be operated under incandescent light, there is no need to run a test under 
florescent light. Moreover, if the system suddenly finds itself operating under fluorescent 
light, a new calibration procedure may be performed. 

[0033] Once the data from the multipHcity of calibration runs have been collected, color 
calibrator 44 may attempt to associate the patch data for each color chart 56 with the viewed 
data. Once this is done, color calibrator 44 may attempt to associate the viewed data with 
the appropriate one of the N bar code colors. Thus, color calibrator 44 may produce a color 
range database 72 which lists each bar code color and the values produced by camera 54 
upon viewing various patches of that color under the various conditions. 
[0034] Figs. 4A and 4B, to which reference is now made, are schematic illustrations 
useful in understanding how color calibrator 44 may determine the viewed color for each 
patch 58 (i.e. the patch data) from the image data received from camera 54. 
[0035] Fig. 4A shows an exemplary image 80 received from camera 54. As can be seen, 
camera 54 viewed color chart 56 over only a portion of its field of view; moreover, color 
chart 56 is not aligned with axes 82 of camera 54. In accordance with a preferred 
embodiment of the present invention, color calibrator 44 may generate patch data by first 
having a user indicate comers 84 of color chart 56. Color calibrator 44 may determine the 
equation of two parallel lines 86A and 86B (Fig. 4B) running through two parallel 
neighboring pairs of comers 84. Color calibrator 44 may then divide the portion of lines 
86A and 86B between comers 84 into twice the number of patches expected to be along the 
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line. For example, if color chart 56 is a 10x10 square of patches 58, then color calibrator 44 
may divide the portion of tfie line into 20 sections, marking each section with a division 
mark 88. 

[0036] For each odd number division mark 88, color calibrator 44 may then drop a line 
90 connecting the corresponding odd numbered division marks 88 and may divide line 90 
into twice the number of patches as expected to be in that direction. For the example above, 
color calibrator 44 may divide line 90 into 20 sections and may select the odd numbered 
ones, here labeled 92. As can be seen in Fig. 4A, marks 92 fall out generally in the middle 
of patches 58. * 

[0037] Color calibrator 44 may then select the RGB values present at the location of 
marks 92 and may associate these values with the bar code color that was supposed to be 
printed for that patch. In other words, if color chart generator 42 (Fig. 2) listed that color 5 
was printed for patch 7 in row 2, then color calibrator 44 may associate the RGB value at 
the 15* division mark 92 of the 3"^ line 90 with color 5 and may store this association in 
color range database 72. 

[0038] Fig. 5, to which reference is now made, is a plot of color range database 72 for an 
exemplary calibration in a system having seven (7) bar code colors (i.e. N=7). The axes of 
Fig. 5 are the three color axes, red, green and blue, and plotted are the values of the received 
data. As can be seen, the color space is a cube 73 of size 255x255x255 and the seven 
viewed colors hardly fill cube 73. Instead, each color 74 fills a very small volume of cube 
73. As a result, there are portions 76 in the color space not associated with any of the seven 
colors. 

[0039] Color range database 72 may optionally be provided to color classifier 46 (Fig. 
1), described in more detail hereinbelow with respect to Figs. 7 and 8, to generate color 
calibration lookup table (LUT) 94 which associates each point in color space 73 with only 
one of the N bar code colors of the color code. Reference is now briefly made to Figs. 6A, 
6B, 6C and 6D, which show the output of color classifier 46 for the exemplary database 72 
plotted in Fig. 5. Figs. 6 A, 6B, 6C and 6D are slices of cube 73 taken along the red axis, 
where red=10 for Fig. 6A, 150 for Fig. 6B, 200 for Fig. 6C and 250 for Fig. 6D. As can be 
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seen, the entire color cube 73 is now filled and each data point is associated with only one 
bar code color. Thus, color space 73 is now divided into seven sections, one per bar code 
color. 

[0040] Figs. 7 and 8, to which reference is now made, show the method performed by 
color classifier 46 to produce color calibration LUT 94. In general, color classifier 46 may 
assign each and every point in the RGB color space 73 with a bar code color label, based on 
a set of inputted RGB values with their corresponding bar code color labels. This is actually 
a classic classification problem. Given a set of training vectors, i.e. a set of samples and 
their corresponding labels, provide a label to an arbitrary test point x. 
[0041] Color classifier 46 may operate similar to the well-known k-nearest-neighbor 
algorithm, such as can be found in the book Pattem Classification by Duda, Hart and Stork, 
pp. 4 -13, 174- 187. 

[0042] The standard k-nearest-neighbor algorithm may classify a test point x by 
assigning it the label most frequently represented among the k nearest samples; in other 
words, a decision may be made by examining the labels on the k nearest neighbors and 
taking a vote. The k-nearest-neighbor query may start at the test point x and may grow a 
spherical region until it encloses k training samples, and it may label the test point x by a 
majority vote of these samples. 

[0043] In the present invention, there may be more than one training vector per RGB 
value, and their labels may not be identical due to noise. Hence, all RGB values in the RGB 
color space may be assigned a bar code color label, including RGB values that are used as 

training vectors. 

[0044] Moreover, there may be an uneven amount of training vectors for the bar code 
colors and the results may be biased. In order to overcome this flaw, color classifier 46 may 
pre-calculate a weight per bar code color, so that the histogram of the training vector's bar 
code colors may be equalized. For example, suppose there were 5 bar code colors, of which 
four colors have 100 training vectors and one has 95 training vectors. In this example, the 
training vectors for the four bar code colors may receive a weight of 1 , while the training 
vectors of the fifth bar code color may receive a weight of 1.053 (100/95). 
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[0045] Color classifier 46 may initially find (step 100) a RGB value's k-nearest- 
neighbors, for example, k=10, but with some modifications. The following steps (shown in 
detail in Fig. 8) describe this process in detail. In any case, once at least k neighbors are 
found, the rest of the steps are skipped. 

1. (Step 102) Within a MIN__RADIUS (for example, MIN_RADIUS=3) 
cube of the current test point color classifier 46 may take all (even more 
than k) of the training vectors. 

2. (Step 104) Within a MAX^RADIUS (for example, MAX_^RADIUS=10) 
sphere of the current test point, color classifier 46 may take a minimum of 
k nearest neighbors. If there are two or more training vectors at equal 
distance from the test point, color classifier 46 may include all of them. 

3. (Step 106) Color classifier 46 may search the entire color space 73 for 
k/2 neighbors. 

[0046] Given the k nearest neighbors, color classifier 46 may select (step 108 of Fig. 7) 
the bar code color with the highest score, based on a weighted vote among the k neighbors. 
Color classifier 46 may sum the number of training vectors of a particular bar code color 
and multiply the sum by that color's weight to produce the score for that bar code color. 
[0047] Color classifier 46 may now determine (step 110) if the majority is distinct. In 
other words, if the selected bar code color is higher by a significant amount, that color has a 
distinct majority. In one embodiment, the highest score may be greater by at least 1 than the 
second highest score to be distinct. If the majority is distinct, color classifier 46 may 
conclude (step 1 1 1) by setting the label for the current RGB value to that of the majority bar 
code color. Otherwise, color classifier 46 may repeat the process for 2k and k/2 training 
vectors (step 112) and may choose the most distinct of the k, 2k and k/2 majority colors 
(steps 114 and 116). 

[0048] Finally, color classifier 46 may set the bar code color for the current RGB value 
to the majority bar code color produced from steps 110 or 116. Color classifier 46 may 
repeat the process for all of the RGB values of color space 73 and may produce color 
calibration LUT 94 as a result. 
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[0049] It will be appreciated that color calibration LUT 94 may be produced by many 
methods; the method described above with respect to Figs. 7 and 8 is only one embodiment. 
Other embodiments are incorporated within the present invention. 

[0050] It will further be appreciated that calibration system 40 (Fig. 2) may produce one 
color calibration LUT 94 or multiple LUTs. In one embodiment, calibration system 40 may 
produce a single LUT 94 for all environmental conditions. In another embodiment, 
calibration system 40 may produce one LUT 94 for each expected lighting condition. In the 
latter embodiment, identifier 39 (Fig. 1) may change LUTs whenever the lighting condition 
may change. This change may occur automatically or in response to a user indication. 
[0051] While certain features of the invention have been illustrated and described herein, 
many modifications, substitutions, changes, and equivalents will now occur to those of 
ordinary skill in the art. It is, therefore, to be understood that the appended claims are 
intended to cover all such modifications and changes as fall within the true spirit of the 
invention. 
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